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*File 431: There is no data missing. UDs have been adjusted to reflect 

the current months data. 

File 442:AMA Journals 1982-2000/Nov B2 

(c)2001 Amer Med Assn -FARS/DARS apply 
*File 442: There is no data missing. UDs have been adjusted to reflect 
the current months data. See Help News442 for details. 
File 77 Conference Papers Index 1973-2001/Mar 

(c) 2001 Cambridge Sci Abs 
File 266:FEDRIP 2001/Apr 

Comp & dist by NTIS, Intl Copyright All Rights Res 
File 444: New England Journal of Med. 1985-2001/Apr W4 
(c) 2001 Mass. Med. Soc. 
*File 444: UDs have been adjusted to reflect the current months data. 
There is no data missing. 
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S AVERILL AND CLASSIFICATION AND RISK AND SEVERITY 

56 AVERILL 
117017 CLASSIFICATION 
319024 RISK 
83648 SEVERITY 

SI 10 AVERILL AND CLASSIFICATION AND RISK AND SEVERITY 



TYPE S1/3,K/1-10 

»>KWIC option is not available in file(s): 77 

1/3, K/l (Item 1 from file: 442) 

DIALOG (R) File 442:AMA Journals 

(c)2001 Amer Med Assn -FARS/DARS apply. All rts. reserv. 
00115041 

COPYRIGHT American Medical Association 2000 



Sustaining the Implementation of an Evidence-Based Guideline for 
Bronchiolitis (ARTICLE ) 

PERLSTEIN, PAUL H.; KOTAGAL, UMA R. ; SCHOETTKER, PAMELA J. ; ATHERTON, 

HARRY D.; FARRELL, MICHAEL K. ; GERHARDT, WENDY E . ; ALFARO, MARY PAT 
American Journal of Diseases of Children 
Oct, 2000; Article: tzplOOl 
LINE COUNT: 00546 

. . . 90% of the population in the single All Patient Refined Diagnosis 
Related Group (APR-DRG) classification of 096 that is common for patients 
with asthma or bronchiolitis . 13/ There was no difference between different 
years of admission in this APR-DRG 096 predominance. Disease severity 
rating, based on number, type, and mix of comorbid diagnoses, was mildly 
ill for all... in the emergency department and diagnosed as having asthma. 

Because of uncertainties . about the true risk of misdiagnosing 
asthma as bronchiolitis in an individual patient,, the guideline used in our 
study... SA. The distinction between cost and charges. Ann Intern Med. 
1982;96:102-109. 
13. 

Averill R, Goldfield N, Steinbeck B, Muldoon J, Beaudry P. All 



1/3, K/2 (Item 2 from file: 442) 

DIALOG (R) File 442:AMA Journals 

(c)2001 Amer Med Assn -FARS/DARS apply. All rts. reserv. 
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00107874 

COPYRIGHT American Medical Association 1998 

Functional Impairment and Co-occurring Psychiatric Disorders in Medically 
Hospitalized Men (ARTICLE) 

BOOTH, B REN DA M.; BLOW, FREDERIC C. ; COOK, CYNTHIA A. LOVELAND 
Archives of Internal Medicine 
July 27,, 1998; Original: tzil551 
LINE COUNT: 00853 

indicates the importance of identification and treatment of 
co-occurring psychiatric disorders in this high- risk and clinically 
challenging group of patients. Arch Intern Med. 1998;158:1551-1559 
... of the National Institute of Mental Health Epidemiologic Catchment Area 
(ECA) study demonstrated that the risk of having a recent psychiatric 
disorder was increased 41% by the presence of a chronic... 

... disorders to functional impairment. We conducted these analyses 
controlling for patient sociodemographics, medical diagnosis, and severity 
of medical illness. 
RESULTS 
SAMPLE DESCRIPTION 

One thousand seven male inpatients from the 3 VAMCs... 

...marital status, and medical diagnostic group (Figure). Early exploratory 
analyses also included the measure of severity of illness, the maximum 
CSI score that had been completed on only a subset of subjects (n = 603) 
because of resource limitations. However, illness severity was not 
independently associated with any measures of functional status, other than 
Physical Functioning. Theref ore ... status , and medical diagnosis, and also 
for the remaining diagnostic groups (Table 4) . As before, severity of 
medical illness was independently associated with only Physical 
Functioning, so the analyses were conducted. . . found only minimal 
correlations between the SF-36 subscales and our objective measure of 
illness severity , the maximum CSI (all Pearson r<0.1), suggesting that 
the acuteness of the medical ... Med Care. 1988;26:724-735. 
30. 

Horn SD, Sharkey PD, Buckle JM, Backofen JE, Averill RF, Horn RA. 
The relationship between severity of illness and hospital length of stay 
and mortality. Med Care. 1991;29:305-317. 
31. 

US Dept of Health and Human Services. The International 
Classification of Diseases, Ninth Revision, Clinical Modification. 
Washington, DC: US Government Printing Office; 1980. 
32. 

Thomas JW, Ashcraft MLF. Measuring severity of illness: six 
severity systems and their ability to explain cost variations. Inquiry. 
1991;28:39-55. 
33. 

Pedhazur... during the index hospitalization at the same time as the 

Q-DIS. 

We measured illness severity to avoid confounding the decreased 
functioning for seriously medically ill patients with any impairment 
associated with the co-occurring psychiatric disorder. We used an 
independent assessment of medical illness severity , the Computerized 

Severity Index (CSI) .30/ Using clinical data abstracted from medical 
records, the CSI provides an overall maximum severity score for each 
patient that is based on the severity of all existing disease states. 
Information in the medical record, including multiple International 

Classification of Diseases, Ninth Revision, Clinical Modification 
codes, 31/ disease complications, and objective clinical findings such as 
laboratory values and symptoms are used to generate a severity score from 
1 to 4.30/ The CSI presents several important advantages over other 
measures of disease severity , including score comparability among all 
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individual and combinations of medical diagnoses and improved prediction of 



... was examined after adjusting for age, race (white or nonwhite), marital 
status, medical diagnosis, and severity of illness. First, we tested the 
associations of the separate SF-36 subscales with a... the effect of mood 
and substance abuse disorders and demographics, medical diagnosis, and 
medical illness severity . 



1/3, K/3 (Item 3 from file: 442) 

DIALOG (R) File 442:AMA Journals 

(c)2001 Amer Med Assn -FARS/DARS apply. All rts. reserv. 
00105480 

COPYRIGHT American Medical Association 1997 
The Risks of Risk Adjustment (ARTICLE) 
IEZZONI, LISA I. 

JAMA, The Journal of the American Medical Association 
November 19, 1997; 19: tzjl600 
LINE COUNT: 00813 



The Risks of Risk Adjustment 

Context. — Risk adjustment is essential before comparing patient 
outcomes across hospitals. Hospital report cards around the country use 
different risk adjustment methods. Objectives. — To examine the history 
and current practices of risk adjusting hospital death rates and consider 
the implications for using risk -adjusted mortality comparisons to assess 
quality. Data Sources and Study Selection. — This article examines 
severity measures used in states and regions to produce comparisons of 
risk -adjusted hospital death rates. Detailed results are presented from a 
study comparing current commercial severity measures using a single 
database. It included adults admitted for acute myocardial infarction (n=ll 



... 016), and stroke (n=9407). Logistic regressions within each condition 
predicted in-hospital death using severity scores. Odds ratios for 
in-hospital death were compared across pairs of severity measures. For 
each hospital, z scores compared actual and expected death rates. 

Results. — The severity measure called Disease Staging had the highest c 
statistic (which measures how well a severity measure discriminates 
between patients who lived and those who died) for acute myocardial 
infarction, 0 . . . 

... the measure, MedisGroups, had the highest for pneumonia, 0.85 and 
stroke, 0.87. Different severity measures predicted different 
probabilities of death for many patients. Severity measures frequently 
disagreed about which hospitals had particularly low or high z scores. 
Agreement in identifying low- and high-mortality hospitals between 
severity -adjusted and unadjusted death rates was often better than 
agreement between severity measures. Conclusions. — Severity does not 
explain differences in death rates across hospitals. Different severity 
measures frequently produce different impressions about relative hospital 
performance. Severity -adjusted mortality rates alone are unlikely to 
isolate quality differences across hospitals. JAMA. 1997;278... 
...adjustment for differences in patients 1 risks. Observers generally agree 
that some hospitals treat more higher- risk patients than others and that 
hospitals should not be penalized for accepting risky patients. In 
addition, without risk adjustment, hospitals with poor outcomes can 
argue, "But my patients are sicker. 1 

Despite this general consensus about the need for risk adjustment 
when comparing outcomes, as in publicly released hospital report cards, 3/ 
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the devil is in the details. Determining which risk factors to include 
and how to measure them generates controversy. Practical constraints 
present difficulties. While researchers have devised detailed risk 
adjustors, 4, 5/ these methods are often too expensive to apply in 
large-scale report card efforts for hospitals. Finally, what do risk 
-adjusted outcomes tell us? 

This article reviews issues raised by risk adjustment for publicly 
comparing outcomes across providers. Because most outcomes-based report 
cards to date . . . 

. . . death rates, 6-17/ I concentrate on this scenario, but the general 
principles pertain to risk adjustment in other settings as well. 
WHY RISK ADJUST? 
Hospitals vary, sometimes widely, in their death rates. The rationale 
for risk adjustment is to remove one source of this variation, leaving 
residual differences to reflect quality. . . 

... result from a complex mix of factors: patient outcomes equal 
effectiveness of treatments plus patient risk factors that affect 
response to treatment plus quality of care plus random chance. 

Controlling for patient risk allows us to begin isolating quality 
differences. But s risk adjustment' is a meaningless phrase without first 
answering the question: risk of what? Identical risk factors may have 
different relationships to different outcomes (ie, an attribute suggesting 
high risk of one outcome may indicate low risk of another outcome) . For 
example, when refining diagnosis related groups (DRGs) to improve their 
sensitivity to severity for hospital payment, researchers found that 
medical patients dying within 2 days of admission had relatively low-cost 
hospitalizations . 18/ A risk adjustor that predicts one outcome (eg, 
death) may not predict another outcome (eg, costs) . 
Many. . . 

. . . risks, including age, sex, acute physiological stability, principal 
diagnosis (ie, reason for hospitalization) and its severity , the extent 
and complexity of comorbid illnesses, functional status, psychosocial and 
cultural factors, socioeconomic characteristics... 

. . . Black patients21/ and uninsured patients22/ may receive lower-quality 
hospital care than others. When using risk -adjusted outcomes to evaluate 
quality, adjusting for race or payer could mask these important 
differences . 

WHERE TO START? v 

Most report card efforts trade off detailed, clinical risk 
assessments for logistical feasibility and reasonable cost. In addition, 
designing a method from scratch is... 

... to predict in-hospital mortality for comparing hospital outcomes (Table 
1) . 23-41/ Often called severity measures, their evolution highlights the 
most important practical concern in widespread risk assessment—the data 
source . 

Early Development of Severity Measures 

Researchers began creating tools for systematic severity 
measurement in the 1970s, generally to address local concerns. 42/ For 
instance, the measure called Computerized Severity Index (CSI) descends 
from efforts to help a specific hospital respond to state regulators' 
questions . . . 

. . . Vincent physicians observed morning report of the medical residents, 
noting clinical parameters that drove residents 1 severity 
assessments . 30, 44/ These observations led to MedisGroups ' initial list of 
key clinical findings (KCFs) . 

Medicare's adoption of DRG-based prospective payment in 1983 elevated 
severity concerns to national prominence . 4 6/ While DRG categories 
initially were derived clinically, their developers made... 

. . . patient demographic data; payer information; principal and other 
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diagnoses and procedures coded using the International Classification of 
Diseases, Ninth Revision, Clinical Modification (ICD-9-CM); admission 
source; and discharge disposition (eg... 

... coding of diagnoses not supported by clinical documentation . 48/ In 
addition, studies suggested that important risk factors, especially 
chronic illnesses, are undercoded. 4 9-51/ Nonetheless, despite limited 
clinical information of questionable. . . 

important advantages of uniformity, widespread availability, and 
computer readability. 

Recognizing these benefits, developers of some severity measures 
(eg, Disease Staging39, 40/ and Patient Management Categories [PMCs] 
Severity Scale52/) translated their severity logic into ICD-9-CM codes 
and designed ...files to refine the specifications of their method. 
Quality Concerns and the Next Generation of Severity Measures 
Payment based on the DRG heightened concerns about quality of care, 
which included fears... 

... caring for terminally ill patients. This early HCFA model had serious 
methodological flaws, including inadequate severity adjustment . 54 / 

At the same time, leaders in several states and regions wanted proof 
about . . . 

...Minneapolis, Minn, led by Walter McClure, who viewed clinically credible 
data as essential to valid risk adjustment. McClure argued, s There is no 
other way than to go in and abstract the clinical findings from the chart. 
Let's stop fooling ourselves that we can compare patient severity by 
claims. '55/ The "buy right' program contracted with MedisGroups, 
abstracting hundreds of KCFs from medical records to measure severity . 

Building off the experience in Minneapolis, the state legislature in 
Pennsylvania adopted what became known... 

... in 1986, largely at the urging of local businesses. Section 3 of Act 89 
defined severity as v in any patient, the measurable degree of the 
potential for failure of one or... 

...to develop and test refinements to their method. 

Thus, in the late 1980s, initiatives to risk adjust hospital 
mortality rates often required clinical information from medical records 
for severity adjustment. In addition to the MedisGroups mandates in 
Pennsylvania, Iowa, and Colorado, other initiatives also concentrated on 
collecting clinical data for risk adjustment, including the Cleveland 
Health Quality Choice program, 17/ a similar effort in St Louis, the 
Department of Veterans Affairs Surgical Risk Study, 56/ and programs to 
evaluate coronary artery bypass graft (CABG) mortality in New York... 

... England. 35, 36/ With growing clinical databases, empirical techniques 
could now be used to refine severity measures. In addition, with 
relatively little effort, different models could be derived empirically to 
predict . . . 

. . . MedisGroups versions to predict in-hospital death or length of stay) . 
This new generation of severity measures, therefore, appeared both 
scientifically rigorous and clinically credible. 

Nevertheless, the costs of clinical data... 

... that the cost was not worth the information benefit. Iowa chose a 
discharge abstract-based severity measure called All Patient Refined DRGs 
(APR-DRGs) for risk adjustment. Colorado's MedisGroups program, dissolved 
in 1995, was also replaced by APR-DRGs . 57 .. . 

... 2 million annual price tag. In 1991, California's legislature mandated 
production of 'home-grown' risk -adjusted outcome measures, using the 
state's existing discharge abstract database. 16/ A similar experience... 
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... of the Medicare hospital mortality reports, citing the inadequacy of 
HCFA's discharge abstract-based risk -adjustment method, especially for 
inner-city public hospitals. 7/ Questions about data quality forced 
California . . . 

. . . accuracy, which found striking variations across hospitals in the 
validity and reliability of coding certain risk factors. 13/ Variation in 
coding accuracy explained part of the differences between hospitals that 
were . . . 

...hospital to 74% at a facility considered low mortality . 13/ 

Thus, current initiatives to compare risk -adjusted outcomes across 
hospitals sort into 2 camps: those willing and those unwilling to pay... 

. . . and thus, presumably, power to sway physicians) and the second arguing 
resource constraints. 

HOW DO SEVERITY MEASURES COMPARE? 

Choosing among severity measures is often daunting (Table 1), but 
selecting different approaches produces a patchwork quilt of Dayton 
Employer Coalition recently chose the vendor of the Disease Staging 
severity measure. 58/ Given the differences across severity measures, 
might different measures give different results? 

Despite their widespread use, role in report cards, and potential 
influence in competitive health care marketplaces, severity measures have 
received little external scrutiny. Most publications about specific 
severity methods come from developers, and statistical performance 
measures vary (Table 2). 59-63/ Because patient... 

... Few comparative studies have been performed, 59-62/ and most report on 
versions of the severity measures that are now out-of-date (Table 2). 
Conducting comparative studies is hampered by. . . 

... of a single database containing all data elements in the exact form 
required by different severity measures and the proprietary ownership of 
many measures. 

Our research group has conducted the only extensive, external study 
of several commercial severity measures sold for comparing hospital 
mortality rates. 64-73/ We studied over a dozen approaches... 

. . . Logistic regressions were run within each condition to predict death 
using patient age, sex, and severity scores . 64-71/ Separate models were 
produced for each severity measure, and their statistical performance was 
judged using the c statistic (which measures how well a severity measure 
discriminates between patients who lived and those who died, so that c=0 . 5 



. . . these z scores . 

Predicting In-Hospital Death for Individual Patients 

Statistical Performance. — Potential purchasers of severity measures 
often look first at the statistical performance of severity measures 
summarized by c and R2/ statistics. As shown in Table 3, our c and R2/ 
values were similar to others reported in the literature (Table 2), and no 
severity measure performed best across all conditions . 64-71/ MedisGroups 
produced the highest c and R2 ... expensive than MedisGroups. 

Comparisons of In-Hospital Mortality Predictions. — The next question 
was whether different severity measures predict different likelihoods of 
in-hospital death for the same patients? There are a... 

. . . death68-71/; however, regardless of the approach, the answer to this 
question was yes: different severity measures rated many patients very 
differently. For the analysis presented here, odds ratios (ORs) of death 
calculated by each severity measure were compared (eg, the odds of dying 
predicted by MedisGroups was divided by the... 

...DRGs disagreed on 60.7% of CABG patients. 
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Comparing Hospital Mortality Rates 

Finding differences across severity measures at the level of 
individual patients led to the question: would judgments about whether 
hospitals look particularly good or bad differ using different severity 
measures for risk adjustment? The answer to this question is 
yes — sometimes. 64-67/ As before, there are several different approaches to 
addressing this question. Here, severity -adjusted death rates were used 
the way report cards or health insurers may use the... 

...scores, hospitals were identified at 2 extremes: 

1. The best 10%: hospitals with the lowest severity -adjusted death 
rates (the lowest z scores) . These hospitals could be designated as 
exemplar benchmarks for quality improvement initiatives. 

2. The worst 10%: hospitals with the highest severity -adjusted 
death rates (the highest z scores) . Insurers may choose not to contract 
with these institutions. 

Table 5 shows how often pairs of severity measures agreed about 
which hospitals fell into the best and worst 10% and how hospital rankings 
based on z scores associated with 'raw' mortality rates (unadjusted for 
age, sex, or severity ) agree. The immediate impression is that severity 
measures often flagged different hospitals than unadjusted mortality rates, 
but different severity measures also frequently flagged different 
hospitals. No clear pattern emerged, which suggests that discharge abstract 



. . . for stroke) . 

When disagreements occurred, hospitals ranked in the top or bottom 
10% by one severity measure often appeared in the next decile (11% to 20% 
or 81% to 90%) ranked... 

...Disease Staging had 266 pneumonia patients with 20 deaths. 

Interestingly, agreement in flagging hospitals ■ between severity 
-adjusted and unadjusted mortality rates was often better than agreement 
between pairs of severity measures . 64-67/ For example, MedisGroups and 
Disease Staging agreed on only 3 of the... 

... for AMI , while MedisGroups and unadjusted rankings agreed on 6 
hospitals. Therefore, one set of severity -adjusted findings was not 
obviously better than another or than unadjusted rankings. For each pair of 
severity measures, k statistics were calculated based on whether 
individual hospitals were flagged as among the... 

... greater than expected by chance. 80/ The k values showed fair to 
excellent agreement among severity measures in flagging hospitals (Table 
5; k values are not presented for CABG because the. . . 

. . . suggest that individual hospitals could care greatly about which 
mortality rates are examined—unadjusted vs severity -adjusted rates—and 
about which severity measure is used. While severity measures agreed 
about fragging hospitals more often than chance, rankings for some 
hospitals differed dramatically across severity measures. A logical, 
albeit worrisome, conclusion is that wise hospitals should *shop around' 
for the severity measure that shows them to the best advantage, although 
this measure could vary by condition. 

Choosing among different severity measures requires consideration 
of a variety of factors. Despite their statistical performance, compelling 
reasons argue for diagnosis coding. 48/ Additionally, to draw meaningful 
conclusions about quality based on severity -adjusted death rates, one 
must adjust only for preexisting conditions but not those arising late... 
. . . for dying patients — as a hospital response to discharge abstract-based 
report cards . 

WHAT DO RISK -ADJUSTED HOSPITAL DEATH RATES MEAN? 
Hospitals vary in their unadjusted death rates. According to our work 
and the findings of others, 35, 82, 83/ severity fails to explain these 
differences fully. As shown in California, differences across hospitals in 
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accuracy of data can account partially for discrepancies in risk -adjusted 
death rates, using measures based on discharge abstracts . 13/ The central 
question remains unresolved: does severity adjustment isolate that 
residual quantity, namely quality-of-care differences, across hospitals?54 / 
A definitive . . . 

... expected death rates. Using clinically detailed severity4/ and quality 
measures, Kahn et al85/ found that risk -adjusted mortality rates were 
significantly related to explicit judgments about process of quality of 
care for 4 of 5 conditions studied. Using PMCs for risk adjustment, 
Thomas et al86/ found that hospital mortality performance was significantly 
related to quality of... 

...rather than low-mortality outlier hospitals . 31/ 

Other studies failed to find any relationship between risk -adjusted 
mortality rates and hospital quality. Using the same rigorous severity 
and quality measures as Kahn et al,85/ Park et al88/ found that hospitals 
flagged. . . 

. . . calculated by the US Department of Veterans Affairs using discharge 
abstract-type data. 89/ 
WHY RISK ADJUST? 
Report cards comparing death rates across hospitals are unlikely to 
vanish. Nonetheless, uncertainty persists about what risk -adjusted 
hospital death rates really mean. In addition, research suggests that 
different risk adjustors can produce different judgments about a 
hospital ! s mortality performance . 64-67/ No studies are available, or 
likely, to tell us which risk adjustor is best, especially for isolating 
quality differences. The initial question, therefore, returns: given these 
vagaries, why risk adjust? The major reason is that, however imperfect, 
there is no other way to begin... 

. . . and other clinicians about using outcomes information to motivate 
quality improvement. 

Another compelling reason to risk adjust is to avoid penalizing 
providers who treat high- risk patients. As report cards naming individual 
hospitals and physicians increasingly make the front pages of . . . 
. . . evidence suggests that some physicians and hospitals turn away patients 
they view as especially high risk .90,91/ Ensuring clinically credible 

risk adjustment could guard against this. Most current severity 
measures, however, do not include all patient characteristics that increase 

risk , such as physical functional status, patients' preferences for care 
and outcomes, cultural factors, and socioeconomic characteristics . 19/ Risk 

must be assessed within subpopulations of patients (eg, racial and ethnic 
minorities, medically indigent persons) without bias. Even with clinically 
derived risk adjustment, some hospitals, like the Cleveland Clinic, will 
argue that their patients are dif f erent . 92/ 

Designing a clinically reasonable but logistically feasible risk 
adjustment method is challenging and demands trade-offs. In today's highly 
charged, competitive environment/ criticisms of risk adjustment are 
inevitable and often appropriate. The accumulated evidence suggests that 
strong inferences about quality should not be made based on risk -adjusted 
mortality rates alone. However, the risk of not risk adjusting is that 
inf ormation--albeit imperfect — will be summarily dismissed. Opportunities 
will be lost for. . . 

... Health Care Policy and Research, Rockville, Md, under grant No. R01 
HS06742. 

The 4-year severity study could not have happened without the 
creativity, energy, and determination of Michael Shwartz, PhD. . . 

...1968. 

5. Iezzoni LI, Shwartz M, Moskowitz MA, Ash AS, Sawitz E, Burnside S. 
Illness severity and costs of admissions at teaching and nonteaching 
hospitals. JAMA. 1990;264:14 26-14 31. 
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6... Health Planning and Development; 1996. 

14. Iezzoni LI, Shwartz M, Restuccia J. The role of severity 
information in health policy debates. Inquiry. 1991;28:117-128. 

15. Iezzoni LI, Greenberg LG. Widespread assessment of risk 
-adjusted outcomes: lessons from local initiatives. Jt Comm J Qual Improv. 
1994;20:305-316. . . 

...specific comorbidities and complications. Med Care. 1995;33:806-827. 

19. Iezzoni LI. Dimensions of risk . In: Iezzoni LI, ed. Risk 
Adjustment for Measuring Healthcare Outcomes. 2nd ed. Chicago, 111: Health 
Administration Press; 1997. 
20. Calkins. . . 

...JAMA. 1994;15:1169-1174. 

22. Burstin HR, Lipsitz SR, Brennan TA. Socioeconomic status and 
risk for substandard medical care. JAMA. 1992;268:2383-2387. 

23. Knaus WA, Wagner DP, Draper EA, et al . The APACHE III prognostic 
system: risk prediction of hospital mortality for critically ill 
hospitalized adults. Chest. 1991;100:1619-1636. 
24 . . . 

...Intern Med. 1993;118:753-761. 

25. Horn SD, Sharkey PD, Buckle JM, Backofen JE, Averill RF, Horn 
RA. The relationship between severity of illness and hospital length of 
stay and mortality. Med Care. 1991;29:305-317. 
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computerized severity index. Qual Rev Bull. 1992;18:4 4-52. 

27. Daley J, Jencks S, Draper D. . . 
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1990;264:2768-2774. 

32. Hannan EL, Kilburn H. . . 
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. . . Retrospective analysis of the 1992 Massachusetts state hospital 
discharge database, using prospectively developed and validated risk 
-stratification models. Setting . --All nonfederal hospitals in 

Massachusetts. Subjects. — Of all adult hospitalizations where an... 

. . . regression models were constructed using split-halves validation to 
adjust for differences in age, sex, severity of illness, diagnosis, 
discharge status, and payer. Separatemodels were constructed for those 
younger than 65 . . . 

... R/2/L/=0.89 for the validation set). Significant covariables affecting 
LOS included age, severity of principal illness, comorbidity, reason for 
admission, and discharge status (P<.001 for each). Among... 

...12.1% in patients aged <greater than or equal to>65 years [P<.05]). Age, 
severity of principal diagnosis, comorbidity, and reason for admission 
significantly influenced mortality <P<.001). After controlling... 
. . . for ICU resource consumption, is not reduced under managed care when 
adjusted for differences in severity of illness. Wealso explored whether 
insurance status was predictive of hospital mortality and whether 
differences . . . 

... sex, and ethnicity; diagnosis related group (DRG) assignment; up to 9 
principal and secondary International Classification of Diseases, Ninth 
Revision, Clinical Modification ( ICD-9-CM) /5/ diagnosis codes and 9 ICD. . . 

. . .Medicare-sponsoredmanaged care plan. 

Covariables included payer (as defined by groups 1-4), age, sex, 
severity of principal diagnosis, comorbidity, admission type (medical, 
elective surgical, or emergent/urgent surgical), and discharge... . 

... or equal to>75 years), and discharge status was recorded as alive or 
dead. 

The severity of principal diagnosis was determined using Disease 
Staging (The MedStat Group, Ann Arbor, Mich). Disease... 

... a software product developed by Gonnella et al/7/ to attribute levels, 
or stages, of severity to hospital discharge diagnoses. It was first 
reported in 1984 and has been extensively validated. . . 

... of predefined, mutually exclusive stages of increasing body system 
involvement: stage 1, problems of minimal severity with no complications; 
stage 2, problems limited to an organor system with substantially increased 
risk of complications; and stage 3, generalized, multiple-site 
involvement and poor prognosis. Each stage is... the traditional insurance 
groups (1 and 3) with respect to demographic characteristics, clinical 
characteristics (including severity of illness and comorbidity), ICU LOS, 
and hospital mortality. Statistical significance was assessed using the... 
and the validation set (R/2/=0.83; R/2/L/=0.89) (Figure 2). Severity of 
principal diagnosis, comorbidity, admission type, and discharge status 
significantly influenced LOS in both age... 
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... and P=.13 when cross validated against the validation set). As with ICU 
LOS, age, severity of principal diagnosis, comorbidity, and admission 
type all significantly influenced the probability of death (P. . . 

. . .52) . 

COMMENT 

Our study of almost 90,000 ICU admissions demonstrates that, after 
adjusting for severity of illness, patients admitted to an ICU insured 
under managed care incur similar LOS when... in patients younger than 65 
years, these differences were explained by differences in patient age, 
severity of illness, and comorbidity, and no benefit was seen with 
orwithout risk adjustment in the Medicare recipients. 

The discrepancy between our findings and those of Rapoport et . . . 

...choice of a linear rather than geometric model, incorporation of the DRG 
relative weight, and severity adjustment. The DRG assignment has been 
criticized previously as a case-mix adjustment measure because... 

... in resource utilization unaccounted for by the DRG relative weight by 
including additional indicators of severity , but the distribution and 
weights of these variables resulted in little contribution to their 
model. /36/ Remaining unobserved differences in severity may have 
correlated with payer, confounding their results. Ill/ 
The ICU LOS, though crude, has... 

... admissions. However, if managed care organizations reduced ICU costs by 
avoiding the admission of low- severity patients, we would expect the 
severity of illness of managed care patients who were admitted to the ICU 
to be higher . . . 

...acute or critical state. Braveman et al/51/ recent lyreported an analysis 
of appendectomy rates and risk of ruptured appendix in California. They 
found that patients managed under traditional insurance plans were... 
including cross-checking and auditing in order to reduce coding 
inconsistency. /4/ 

Assigning disease and severity via discharge data has been 
criticized previously. However, though not comparable with admission 

severity systems such as Acute Physiology and Chronic Health Evaluation 
(APACHE) III or Mortality Probability Model II when used to predict risk 

for small groups and individuals, classification schemes such as Disease 
Staging compare well when used to adjust for severity across large 
populations . /8 , 63-66/ Furthermore, much of the criticism arises when such 
data and severity systems are used to predict outcome such as mortality 
since certain codes predictive of death... 

...example, for the small but significant mortality benefit of managed care 
where unobserved differences in severity may have been s carried' by 
payer. This would result in the false conclusion that hospitalization... 

... Administration. 1994 Patient Data Tapes. Tallahassee, Fla: Agency for 
Health Care Administration; 1994. 

5. International Classification of Diseases, Ninth Revision, 
Clinical Modification. Washington, DC: Public Health Service, US Deptof 
Health and Human Services; 1988. 

6. Fetter RB, Thompson JD, Averill RA. The New ICD-9-CM Diagnosis 
Related Groups Classification Scheme: Users Manual. New Haven, Conn: 
Health Systems Management Group, School of Organization and Management... 

; . .644. 

8. Iezzoni LI, Shwartz M, Moskowitz MA, Ash AS, Sawitz E, Burnside S. 
Illness severity and costs of admissions at teaching and nonteaching 
hospitals. JAMA. 1 990 ; 264 : 14 26-14 31 . 

9. SysteMetrics Inc. Disease Staging: A Clinically Based Approach to 
Measurement of Disease Severity . Springfield, Va : US Dept of Commerce, 
National Technical Information Service; 1983. 
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ICU patients. Med Care. 1990;28:338-348. 
41 . Rapoport J. . . 
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rehabilitation patients have many complications that warrant 
attention, none has attempted to categorize complications by severity . 
This retrospective cohort study examined the incidence, types, and 
severity of problems that interrupt rehabilitation and themajor risk 
factors for these events. Results: Of 1075 patients, 359 (33.4%) had 
acute medical complications... 

... or fever (17.1%) and by thromboembolic events (16.5%). Logistic 
regression revealed that major risk factors for complications requiring 
transferwere a primary diagnosis of deconditioning or nontraumatic spinal 
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cord injury (adjusted odds ratio, 2.7; confidence interval, 1.8 to 4.2), 
severity of initial disability (adjusted odds ratio, 1.2; confidence 
interval, 1.1 to 1.3... 

... of comorbid conditions (adjusted odds ratio, 1.1; confidence interval, 
1.0 to 1.2). Risk factors for any complication were similar, but there 
was an interaction between comorbidity and the... 

...were severely functionally impaired, the number of comorbidities was not 
as strongly associated with the risk of complications as it was in 
patients who were less functionally impaired. Conclusion: There is a 
complex relationship among the type of underlying medical impairment, 
severity of functional limitation, comorbidity, and unanticipated medical 
or surgical complications thatinterrupt rehabilitation. The interruptions 
vary both in type and in severity . (Arch Intern Med. 1994;154:2185-2190) 
Felsenthal et al/2/ and Parry, /3/ in preliminary studies, characterized the 
quantity and severity of medical diagnoses in rehabilitation patients. 
Unfortunately, these studies were small, andthey failed to distinguish... 

... pulmonary embolus. One thousand seventy-five patients had sufficient 
data for multivariable analysis of the risk factors associated with 
complications. Three hundred fifty-nine had complications and 158 of ' those 
individuals . . . 

... for each variable along with 95% confidence intervals. Male gender 
appeared to confer an increased risk for complications at the unadjusted 
level. Age, race, and previous living arrangement did not appear... 

... 3 displays information about the relationship between functional status 
as measured by the MBI and risk of complications. For complications as a 
whole and the subset of unexpected transfers, the worse the functional 
status of the patient at admission, the greater the risk for an event. 

Table 3 also demonstrates a relationship between number of 
comorbidities and risk of an event. Patients had on average 1.8 
comorbidities. The risk of complications increased most sharply in the 
group of patients in the highest comorbidity category. . . 

... complications, further analysis could not be performed. Logistic 
regression was then used to examine the risk factors for development of 
all complications or complications requiring transfer off rehabilitation. 
Table 5 lists the risk factorsthat remained statistically significant for 
all medical or surgical complications during rehabilitation. Male gender 
remained significant. Those admitted for nontraumatic SCI or deconditioning 
were also at increased risk . 

There was a statistically significant interaction between MBI and the 
number of comorbid conditions for. . . 

. . . person (Table 5) . This • is presumably because those with few chronic 
illnesses are at lower risk for any complication so that the role of 
initial function is increased. 
The interaction may. . . 

. . . functionally impaired (with low MBIs) , the number of comorbidities was 
not significantly associated with the risk of complications. Only in 
those patients who were functioning at a higher level did the number of 
comorbidities appear to play a role asa risk factor. For example, given 
an MBI value of 20, each additionalcomorbidity added no additional risk 
(odds ratio, 1.0 0.8 to 1.2 ). If the value of the MBI... 

...shows the results of the logistic regression for complications requiring 
transfer. As for the broader classification of any complication, the 
presence of nontraumatic SCI or deconditioning was a significant risk 
factor. Functional status was also significant, with a 1.2-fold increase in 
risk (95% confidence interval, 1.1 to 1.3) for each 10-point decline in 
MBI . . . 

. . . database to determine what kinds of complications develop during 
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rehabilitation and who is at most risk for developing them. Most 
rehabilitation patients have medical problems that need evaluation and 
management. Felsenthal. . . 

...and only 1% of patients had a life-threatening event. 

In addition, categorizing complications by severity allowed 
determination of the roles of different kinds of complications. In this 
population, surgical complications and infections dominated overall, with 
cardiac and thromboembolic complications also playing alarge role. 

Risk factors for the development of complications were few: . . . 
status at the time of admission to the rehabilitation service was also 
related to the risk of complication. On the other hand, age was not 
associated with complication risk at the unadjustedor multivariate level. 
One might have anticipated that age would appear to be a major risk 
factor for complications, perhaps at the adjusted level or at least at the 
unadjusted level... 

This study builds on that relationship, demonstrating a clear 
association in a rehabilitation population between risk of complication, 
degree of functional impairment, and number of comorbidities. However, at 
least for the . . . 

...of functional impairment. 

Of note, Pompei et al/12/ have previously described an interaction 
between severity of illness and functional status, in their case,. in the 
opposite direction of what our... 

. . . and a much more severe outcome, one where a synergy between poor 
functional status and severity of illness would be anticipated. In fact, 
in our study, the interaction was no longer... 

. . . the more severe medical complications (those requiring transfer) were 
included. In this latter case, the severity of functional impairment and 
the number- of comorbid conditions were directly related to complication 
risk . Clearly, medical illness and function do interact with one another; 
the mechanism of that interaction. . . 

...rehabilitation centers. In addition, the population wasa general medical 
rehabilitation population. In-depth investigation of risk factors for 
complications might be better served by focusing on a more uniform 
population — hip fracture or stroke patients, for example. Finally, only a 
few risk factors could be investigated, given the nature of the database; 
documenting underlying comorbidities of these... 

...to more accurate predictions of who will develop complications. 

Patients undergoing medical rehabilitation are at risk for 
complications at different levels of severity . Determining the natureof 
those complications and the risk factors for developing them can allow 
providers to intervene early and prevent significant morbidity and. . . 

... decline. This study begins to tease apart the complex relationships 
between comorbidity, functional impairment, and risk ofmedical 
complications . 

The reasons for unanticipated interruption of rehabilitation vary in 
both diagnosis and severity . Thus, (1) research should extend beyond 
cardiac and thromboembolic disease and continue to examine other types of 
complications; (2) future studies should take the severity of comorbidity 
into account; and (3) given the high frequency of complications in this 
setting... J ChronicDis. 1974;27:387-4 04. 

10. 

Horn SD, Sharkey PD, Buckle JM, Backofen JE, Averill RF, Horn RA. 
The relationship between severity of illness and hospital length of stay 
and mortality. Med Care. 1991;29:305-317... 

. . . Three definitions of complications were used. Those definitions were 



16 of 48 



4/22/01 10:29 AM 



DialogClassic Web(tm) 



http://www.dialogclassic.com/264656RB.HTML? 



hierarchical, and in increasing order of severity : (1) all 
complications—a group that included any event that unexpectedly 
interrupted, delayed, or prolonged. . . 

... tested with quadratic equations to rule out nonlinear effects. Because 
the effects were linear, these risk factors were entered into the 
regression as continuous variables. The statistical software used was the 
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ABSTRACT: Measurement of illness severity is required to evaluate 
diagnostic efficiency of physicians, assess quality of care, understand 
utilization of. . . 

... trials, and reimburse hospitals on the basis of output. "Staging" is a 
method for measuring severity of specific, well-defined diseases. Staging 
defines discrete points in the course of individual diseases that are 
clinically detectable, reflect severity in terms of risk of death or 
residual impairment, and posses clinical significance for prognosis and 
choice of therapeutic. . . 

. . . can be efficiently applied to computerized hospital discharge abstracts 
to derive a comprehensive case-mix classification system. Uses of staging 
in epidemiologic, case-mix, and utilization analyses are illustrated for 
diabetes . . . 

DEVELOPMENT of scales measuring severity of diseases is a major challenge 
for the medical profession. Severity is defined here as the likelihood of 
death or residual impairment as the result of a disease, without 
consideration of treatment. Severity scales are useful in evaluating 
diagnostic efficiency of physicians, refining measures of prognosis, 
therapeutic effectiveness... 

... based on the diagnoses of their patients' conditions. In the area of 
health care evaluation, severity scales enable more accurate measurement 
of health status before, during, and after medical intervention and. . . 

... expected to be similar. However, there is no explicitmechanism in the 
DRGs to account for severity of illness within a particular diagnosis. 

Many physicians and hospital administrators fear that their 
institutions . . . 

... for municipal, teaching, and large referral hospitals. Indeed, to the 
extent that substantial variation in severity is observed within any DRG 
category and to the extent that severity is correlated with resource use, 
hospitals face an incentive to admit only the least severe... 

. . . this potential problem by examining technically appropriate patterns of 
inpatient resource use by level of severity within each DRG category. 

This article describes an approach to measuring disease severity , 
called "staging," that can be used to address this problem and many others. 
Disease staging. . . 

. . . application to medical records and for automated application to 
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computerized hospital discharge abstract data. These severity criteria 
cover practically all admissions to the typical hospital so that staging 
represents a comprehensive case-mix classification sytem. It also 
provides a means of assessing appropriateness of admissions and discharges, 
of examining associations between medical resource use and severity of 
specific diseases, and of comparing hospital outputs on both a large-scale 
basis and. . . 

. . . staging is described in this article, as well as the methods used to 
develop the severity criteria. Examples of potential applications of 
staging are presented, focusing on one disease, diabetes mellitus... 

done by physicians, hospitals, health agencies, or insurance 
companies . The problem is compounded by the classification systems used to 
store, retrieve, and analyze health data. Health problems (symptoms, 
physical abnormalities, and. . . 

...disease: location of the problem, manifestations of the medical problem, 
cause of the problem, and severity of the problem. Each disease should be 
defined in terms of some specific organ or... 

. . . did not know, even after study, or did not disclose what disease 
produced the CHF. 

Severity of the pathophysiological changes must be given, when 
severity is defined as risk of death or temporary or permanent 
impairment. This can be done simply through specification of... to renal 
damage. Pneumococcal pneumonia with septicemia would be considered to be a 
different disease ( severity ) than pneumococcal pneumonia complicated by 
either empyema or meningitis. 

May diagnostic labels traditionally used by medical professionals and 
most diagnostic rubrics in the various coding systems (eg, International 
Classification of Disease: Clinical Modification, ninth revision (Ref. 
6)) do not give these types of data... 

... case mix. Patients are classified into inappropriate groups for 
comparison of utilization patterns, and overall severity of disease for 
any group of patients cannot be assessed. 

Disease Staging 

A method called "staging" has been developed for measuring disease 
severity . (Ref. 7.) The concept behind staging is borrowed from clinical 
medicine, specifically oncology. During the... 

...neoplastic diseases, there are discrete "stages" that can be defined and 
detected clinically, reflect the severity of the disease, and have 
clinical significance for prognosis and choice of therapeutic modality. 
This. . . 

...all hospitalized patients. 

In staging, diseases are generically divided into categories of 
increasing levels of severity : stage 1, conditions with no complications 
or problems of minimal severity ; stage 2, problems limited to an organ or 
system, significantly increased risk of complications over stage 1/ stage 
3, multiple site involvement, generalized systemic involvement, poor 
prognosis . . . 

. . . does not depend on actual utilization patterns or on expected response 
to therapy to assess severity . It is based on a conceptual model of the 
disease process itself rather than on. . . 

...from both heuristic and practical viewpoints, to include death as an end 
point of the severity scale. This provides explicit consideration of 
patients who die during the course of their treatments... 

...discharge abstracts. These steps ensured that staging would be useful as 
a comprehensive case-mix classification system that could be applied to 
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large groups of patients. 

Development of Medical Staging Criteria... 

... primary category as deemed appropriate. Each substage should place the 
patient at a significantly higher risk of morbidity or mortality or both 
than the previous substage and should be clinically dif f erentiable . . . 

... other substages. When complications with each primary stage or substage 
cannot be differentiated in their severity , they should be classified as 
equivalent. Only single disease entities, without concern for complications 
arising. . . 

... to translate each stage and substage definition into diagnostic codes, 
using the three primary disease classification systems: ICDA-8, (Ref. 8) 
H-ICDA-2, (Ref. 9) and ICD-9-CM. (Ref... and white blood cell and platelet 
counts. Again, the coded criteria tend to understate the severity of the 
disease in this situation. 

An example set of medical and coded staging criteria ... cases of a 
disease on average? How does utilization of health services vary with 
disease severity ? 

Data 

The data for the empirical analysis presented herein were derived from 
discharge abstracts of... 

. . . and 3 are combined so as to simplify presentation and obtain higher 
frequencies in each severity level to facilitate further breakdowns by 
patient and hospital characteristics. Chi-square tests were performed...* 
Patients listed as emergency admissions on the discharge abstract are more 
likely to have higher severity and die than regular admissions, thus 
providing a validity check on the staging criteria. 

Fewer than one fifth of diabetes cases are operated on, but they have 
substantially higher severity levels, indicating an association between 
severity and resource use. This is also supported by a strong association 
between stage and number. . . 

...on the abstract. 

Finally, Medicare and self-paying patients are revealed to have 
significantly higher severity levels than other groups of patients. In 
the case of Medicare, this is likely to be age related. However, the higher 
severity of self-pay patients may be related to delay in seeking 
appropriate care, so that... 

...provision of ambulatory care. 

Case-Mix Analysis . -- A second application of staging is measurement 
of severity of hospitals 1 case mixes. By case mix, we mean the variety of 
different diseases being. . . 

... mellitus by selected hospital characteristics are presented in Table 3. 
Again, some interesting associations with severity are found. Proprietary 
hospitals are shown to admit less severely ill diabetic patients as 
compared with voluntary and governmental hospitals. Strong associations 
between bed size and severity are observed . Medical school-affiliated 
hospitals tend to admit relatively more severe cases than nonaffiliated. . . 
however, to illustrate that there are significant case-mix differences 
among hospitals with respect to severity for this diagnosis. 

Utilization Analysis. — An important hypothesis regarding hospital 
utilization is that sicker patients... 

. . . varies with selected patient and hospital characteristics within each 
stage. This enables the dimension of severity to be held constant while 
the impact of other factors on length of stay is... 

...stages 3 and 4, which suggests that length of stay becomes more variable 
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with increased severity and also that economic factors may not be 
relevant at higher severity levels. m 

Length of stay breakdowns for diabetes by selected hospital 
characteristics are presented in Table... 

... between patient groups, with respect to the nature of their illnesses, 
are maximized. Such a classification scheme, which must allow for the 
assessment of severity of disease, can then be used for studies of 
hospital utilization and costs, development of... care with respect to 
length of stay, surgery, and overall service intensity . The association 
between severity and resource use provides predictive validity of the 
staging criteria. 

Considerable within-stage variation in resource use remains, much of 
which may be cause by factors other than biologic severity of illness. 
Patient-related factors, such as age, income level, expectations, health 
insurance coverage, family. . . 

historical data files must be interpreted with caution because high 
within-stage variation in actual severity levels may be present. 

staging is relevant to the individual physician because it provides a 
tool for sharpening diagnostic classification . This will become 
particularly relevant in dealing with the impending regulations governing 
hospital admissions and. . . 

...of this process will be greater requirements for documenting and 
justifying hospital admissions and discharges. Severity will play a 
crucial role here and staging can be used as an operational measure. 

In contrast to other existing casetype classification system such as 
the DRGs, an important characteristic of the staging criteria is their 
emphasis ... 

... use principal operating room procedures to classify patients in 
different groups and ignore etiology and severity of the problem. For 
example, patients receiving a coronary bypass procedure are classified 
differently than... 

... disease across institutions and physicians. Perhaps more important is 
that staging enables incorporation of disease severity into the 
prospective payment system. 
CONCLUSIONS 

The utility of the coded staging criteria and the... 

... contribute to hospital costs or resource requirements. Indeed, it is 
specifically focused only on one -- severity of disease. Thus, staging 
should not be asked to provide a complete explanation of 
variation ... 

. . . was first seen, those developing during the hospitalization, and those 
successfully treated. Only by evaluating severity of patients' illnesses 
at various times can meaningful analyses of quality and cost-benefit 
tradeoffs . . . 
CITED REFERENCES: 
...of hospitals. N Engl J Med 1983;308: 1428-1432. 

3. Fetter RB, Thompson JD, Averill RA, et al: The New ICD-9-CM 

Diagnosis Related Groups Classification Scheme: Users Manual. New 
Haven, Conn, Health Systems Management Group, School of Organization 
and Management. . . 

...part I. The conceptual framework. Med Care Rev 1982;39: 1-43. 

6. The International Classification of Disease: Clinical Modification, 
ninth revision. Ann Arbor, Mich, Commission on Professional and 
Hospital Activities... 
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8. International Classification of Diseases: Adapted for Use in the 
United States, eighth revision. Washington, DC, National Center... 

. . .191-201. 

12. SysteMetrics Inc: Disease Staging: A Clinically Based Approach to 
Measurement of Disease Severity , final report for contract 
233-78-3001, submitted to National Center for Health Services 
Research. . . 
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...including outlier payments, in which payment accuracy might be 
improved without better measurement of the severity of illness. There is 
no available measure of severity of illness that would produce a large 
improvement in the accuracy of Medicare payments if... 

...supplement or replace the system of diagnosis-related groups. Evidence 
regarding whether better measurement of severity would substantially 
change the distribution of payments across hospitals is mixed. Considerable 
evidence suggests that... 

...for patients in the same diagnosis-related group varies substantially 
for reasons other than the severity of illness. Despite great demand for 
measures of the quality of care, important technical problems... 

TEXT 

...with payment issues such as teaching, wages, and hospital location; 
tools for measuring variations in severity of illness within 
diagnosis-related groups (DRGs) ; outliers -- i.e., cases that are so 
unusual . . . 

...describe them very poorly; variations in the intensity of care for 
patients with a similar severity of illness; and whether case-mix 
adjustment permits inferences about the quality of care from. . .Variations 
in the severity of illness and the intensity of care within DRGs and 
differences in efficiency probably account... 

...payment system is equitable; if this variation results from factors such 
as differences in the severity of illness within DRGs, for which Medicare 
should pay, then there is a problem. We... of substantial deliberate 
misreporting. The more serious concern regarding data quality is central to 
the % * severity ■ 1 problem: Medicare collects only the uniform 
hospital-discharge data set, which supplies no clinical information... 

...Moskowitz (Ref. 14) have pointed out that diagnoses coded according to 
the current system (International Classification of Diseases, ninth 
revision, Clinical Modification) provide only a limited and ambiguous 
description of a... 
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. . .Variation in the Severity of Illness within DRGs 

Variations in the clinical severity of illness among patients 
assigned to the same DRG result in differences in costs and outcomes. 
Measurements of severity that predict the cost of care may be quite 
different from measurements that predict outcomes... 

...were developed by clinicians, with very little empirical input. A fully 
detailed description of the classification algorithm does not exist. 
Young, the developer of Patient Management Categories, believes that they 
should. . . 

...be assigned to several categories. Neither system has a well-developed 
mechanism for assessing the severity of a case that is assigned to more 
than one category. The classification algorithms for both systems are 
largely in the public domain, but the computer software toThe Severity of 
Illness Index (Ref. 21) assigns a score from 1 to 4 that represents the 
overall severity of the patient ! s illness during the entire hospital 
stay. The overall score integrates seven... 

. . .may now be of largely historic interest because she is focusing on 
developing the Computerized Severity Index. The new index uses discharge 
diagnoses to produce a list of physiologic data that... 

...then integrates these elements in a diagnosis-specific way to produce an 
overall rating of severity within the DRG. This index is now being 
field-tested. The algorithm for the older. . . 

...is scheduled for publication, but the computer software is proprietary. 
Horn has proposed using the severity index within DRGs to refine DRG 
payments . . . 

...The acute physiology and chronic health evaluation (APACHE) system 
assigns a patient an overall severity score {ranging from 0 to 71) based 
on age, a few data on long-term. . . 

...record. Patients are scored independently of diagnosis. Knaus and 
colleagues developed APACHE to predict the risk of death in intensive 
care units, and it has proved to be a strong predictor. . . 

...Ref. 27) uses a special abstraction of physical examination and test 
data to assign a severity score (0 to 4) to the patient; for payment 
purposes and for predicting outcomes, this... 

...200 possibly important items. The way in which the items are integrated 
to yield a severity score is not diagnosis-specific. The definitions of 
items and the algorithms for scoring are... 

...First, to the extent that a severity measure confirms current beliefs 
that teaching hospitals, ^disproportionate-share' 1 hospitals, and urban 
hospitals have more... 

...Second, although the severity of illness is clearly an important 
determinant of costs in individual cases, differences in severity may 
well average out at the hospital level. Horn et al. (Ref. 29,30) have 
argued that interhospital differences in severity account for a large 
part of the difference between hospital costs and prospective payments, but 



...and work at HCFA (using DRGs revised by staging) have found that 
interhospital differences in severity beyond those measured by DRGs 
account for little of the observed interhospital differences in cost... 

. . .Assessment Commission; the commission takes a fairly cautious view of 
the prospects for measures of severity (Ref. 32... 

...costs, and to collect the data necessary to set payment weights for 
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various levels of severity ■ Experience suggests that 30 to 42 months 
would elapse from the time the project began... 

...In summary, no single solution is in sight for adjusting payments for 
differences in severity within DRGs. Research suggests that we can make 
progress through innovations using currently available data. . . 

...of medically appropriate care provided to patients in the same DRG who 
have a similar severity of illness. Distinguishing this medically 
appropriate variation from variations due to unnecessary, inefficient, or 
inadequate . . . 

...care and the goals of care set an absolute limit on how well measures of 
severity can predict cost and, in many cases, on how well they can 
predict outcomes. On... 

...pay for the former and discourage the latter. 
Variations among Patients 

Patients with a similar severity of illness and in the same DRG 
receive care that varies in intensity for a stays, whereas others are 
treated during a single, longer stay. For example, neither the severity 
of illness nor the nature of the illness fully predicts the amount of care 
that. . . 

...variation in intensity of care among hospitals is likely to remain even 
after the best severity adjustments to DRGs, although better ways to 
describe the purpose of admission could reduce this... Using case-mix 
adjustment to measure the quality of care requires assuming or proving that 

risk -adjusted differences in outcomes result from differences in the 
quality of care. This hypothesis is... 

. . . Severity adjustments that predict outcome are not identical to those 
that predict cost. Because every hospitalization... Risk -adjusted outcomes 
are perhaps best viewed today as screening tools that may suggest where 
problems . . . 

...become an extremely active area of research with a strengthening 
quantitative base. The problem of severity of illness, widely used almost 
as a synonym for * v case-mix adjustment,'' is only part... 

...refining Medicare's prospective payment system. It is ironic that as the 
scientific base regarding severity measurement for payment is growing, 
the public may press for application to the area of 
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Arbor, Mich.: Department of Health Services... 
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Interhospital differences in severity of illness: problems for 
prospective payment based on diagnosis-related groups (DRGs) . N Engl J 
Med 1985; 313:20-4. 

30. Horn SD, Sharkey PD, Chambers AF, Horn RA. Severity of illness within 
DRGs: impact on prospective payment. Am J Public Health 1985; 
75:1195-9. 

31. Berman RA, Green J, Kwo D, Safian KF, Botnick L. Severity of illness 
and the teaching hospital. J Med Educ 1986; 61:1-9. 

32. Fed... 

...Laughlin A, Margolis IB, Wise L. Source of admission and cost: public 

hospitals face financial risk . Am J Public Health 1986; 76:696-7. 
37. Garber AM, Fuchs VR, Silverman JF. . . 

...bedsize. Waltham, Mass.: Brandeis University Health Policy Center, 1985. 

39. Bowen OK. DRG refinement: outliers, severity of illness, and 

intensity of care: report to Congress. Washington, D.C.: Department of 
Health. . . 

...Medicare. Health Care Financ Rev 1982; 3(3):41-73. 
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...including outlier payments, in which payment accuracy might be 
improved without better measurement of the severity of illness. There is 
no available measure of severity of illness that would produce a large 
improvement in the accuracy of Medicare payments if... 

...supplement or replace the system of diagnosis-related groups. Evidence 
regarding whether better measurement of severity would substantially 
change the distribution of payments across hospitals is mixed. Considerable 
evidence suggests that... 

...for patients in the same diagnosis-related group varies substantially 
for reasons other than the severity of illness. Despite great demand for 
measures of the quality of care, important technical problems... 

TEXT 

. . .with payment issues such as teaching, wages, and hospital locations- 
tools for measuring variations in severity of illness within 
diagnosis-related groups (DRGs); outliers — i.e., cases that are so 
unusual . . . 

...describe them very poorly; variations in the intensity of care for 
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patients with a similar severity of illness; and whether case-mix 
adjustment permits inferences about the quality of care from. . .Variations 
in the severity of illness and the intensity of care within DRGs and 
differences in efficiency probably account... 

...payment system is equitable; if this variation results from factors such 
as differences in the severity of illness within DRGs, for which Medicare 
should pay, then there is a problem. We... of substantial deliberate 
misreporting. The more serious concern regarding data quality is central to 
the severity fl problem: Medicare collects only the uniform 
hospital-discharge data set, which supplies no clinical information. . . 

...Moskowitz (Ref. 14) have pointed out that diagnoses coded according to 
the current system (International Classification of Diseases, ninth 
revision, Clinical Modification) provide only a limited and ambiguous 
description of a... 

...Variation in the Severity of Illness within DRGs 

Variations in the clinical severity of illness among patients 
assigned to the same DRG result in differences in costs and outcomes. 
Measurements of severity that predict the cost of care may be quite 
different from measurements that predict outcomes... 

...were developed by clinicians, with very little empirical input. A fully 
detailed description of the classification algorithm does not exist. 
Young, the developer of Patient Management Categories, believes that they 
should. . . 

...be assigned to several categories. Neither system has a well-developed 
mechanism for assessing the severity of a case that is assigned to more 
than one category. The classification algorithms for both systems are 
largely in the public domain, but the computer software toThe Severity of 
Illness Index (Ref. 21) assigns a score from 1 to 4 that represents the 
overall severity of the patient's illness during the entire hospital 
stay. The overall score integrates seven. . . 

. . .may now be of largely historic interest because she is focusing on 
developing the Computerized Severity Index. The new index uses discharge 
diagnoses to produce a list of physiologic data that... 

...then integrates these elements in a diagnosis-specific way to produce an 
overall rating of severity within the DRG. This index is now being 
field-tested. The algorithm for the older. . . 

...is scheduled for publication, but the computer software is proprietary. 
Horn has proposed using the severity index within DRGs to refine DRG 
payments . . . 

...The acute physiology and chronic health evaluation (APACHE) system 
assigns a patient an overall severity score (ranging from 0 to 71) based 
on age, a few data on long-term. . . 

...record. Patients are scored independently of diagnosis. Knaus and 
colleagues developed APACHE to predict the risk of death in intensive 
care units, and it has proved to be a strong predictor... 

...Ref. 27) uses a special abstraction of physical examination and test 
data to assign a severity score (0 to 4) to the patient; for payment 
purposes and for predicting outcomes, this... 

...200 possibly important items. The way in which the items are integrated 
to yield a severity score is not diagnosis-specific. The definitions of 
items and the algorithms for scoring are... 

...First, to the extent that a severity measure confirms current beliefs 
that teaching hospitals, ^disproportionate-share 1 ' hospitals, and urban 
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hospitals have more... 

...Second, although the severity of illness is clearly an important 
determinant of costs in individual cases, differences in severity may 
well average out at the hospital level. Horn et al. (Ref. 29,30) have 
argued that interhospital differences in severity account for a large 
part of the difference between hospital costs and prospective payments, but 

...and work at HCFA (using DRGs revised by staging) have found that 
interhospital differences in severity beyond those measured by DRGs 
account for little of the observed interhospital differences in cost... 

...Assessment Commission; the commission takes a fairly cautious view of 
the prospects for measures of severity (Ref. 32... 

...costs, and to collect the data necessary to set payment weights for 
various levels of severity . Experience suggests that 30 to 42 months 
would elapse from the time the project began. . . 

...In summary, no single solution is in sight for adjusting payments for 
differences in severity within DRGs. Research suggests that we can make 
progress through innovations using currently available data... 

...of medically appropriate care provided to patients in the same DRG who 
have a similar severity of illness. Distinguishing this medically 
appropriate variation from variations due to unnecessary, inefficient, or 
inadequate . . . 

...care and the goals of care set an absolute limit on how well measures of 
severity can predict cost and, in many cases, on how well they can 
predict outcomes. On. . . 

...pay for the former and discourage the latter. 
Variations among Patients 

Patients with a similar severity of illness and in the same DRG 
receive care that varies in intensity for a stays, whereas others are ^ 
treated during a single, longer stay. For example, neither the severity 
of illness nor the nature of the illness fully predicts the amount of care 
that. . . 

...variation in intensity of care among hospitals is likely to remain even 
after the best severity adjustments to DRGs, although better ways to 
describe the purpose of admission could reduce this... Using case-mix 
adjustment to measure the quality of care requires assuming or proving that 

risk -adjusted differences in outcomes result from differences in the 
quality of care. This hypothesis is... 

... Severity adjustments that predict outcome are not identical to those 
that predict cost. Because every hospitalization... Risk -adjusted outcomes 
are perhaps best viewed today as screening tools that may suggest where 
problems . . . 

...become an extremely active area of research with a strengthening 
quantitative base. The problem of severity of illness, widely used almost 
as a synonym for vv case~mix adjustment, 1 ' is only part... 

...refining Medicare's prospective payment system. It is ironic that as the 
scientific base regarding severity measurement for payment is growing, 
the public may press for application to the area of 
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...identifying transfer patients. 

Assignment to Diagnosis-Related Groups 

The original diagnosis-related group (DRG) patient- classification 
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scheme was used to account for differences in patient case mix (Ref . 19) . 
All patient ... its standard metropolitan statistical area to derive 
equalized direct and full costs for each hospital. 
Classification of Teaching Hospitals 
Instead of having teaching intensity defined through a simple head 
count of . . . 

...least 10 residents in internal medicine and family practice combined. 
These criteria resulted in the classification of 15 hospitals as major 
teaching and 51 as minor teaching hospitals. Although based on a subjective 
set of criteria, this classification scheme was intended to account for 
both the depth and breadth of teaching intensity. 

Measuring. . .nonteaching settings. It has been suggested that DRGs do 
not fully capture the differences in severity of case mix between 
teaching and nonteaching hospitals (Ref. 20) . The question of whether 
another patient- classification scheme providing a more sensitive measure 
of severity offers a more appropriate mechanism for comparing costs among 
categories of teaching hospitals hinges on. . . 

...date, this has not been established, in spite of the widespread belief 
that differences of severity within DRGs are important between teaching 
and nonteaching hospitals for high- risk DRGs. Insofar as our study 
included only Medi-Cal patients, a much smaller proportion of whom fall 
within high- risk DRGs than of Medicare patients, the effect of any such 
severity differences is likely to be less pronounced. 
Implications 

This study included only Medi-Cal acute... 
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...0.14 and 0.15 for Lake and Geauga counties to 0.29 and 0. 47 for 
Cuyahoga and Lorain counties, respectively ... and the results of others 
(Ref. 5) have shown no differences in two-year neurodevelopmental outcomes 
after the use of surfactant 
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ABSTRACT: Measurement of illness severity is required to evaluate 
diagnostic efficiency of physicians, assess quality of care, understand 
utilization of health services, design clinical trials, and reimburse 
hospitals on the basis of output. "Staging" is a method for measuring 
severity of specific, well-defined diseases. Staging defines discrete 
points in the course of individual diseases that are clinically detectable, 
reflect severity in terms of risk of death or residual impairment, and 
posses clinical significance for prognosis and choice of therapeutic 
modality. Medical staging criteria have been developed for 420 diagnoses 
and converted into "coded" criteria for the major diagnostic coding 
systems. Coded criteria can be efficiently applied to computerized hospital 
discharge abstracts to derive a comprehensive case-mix classification 
system. Uses of staging in epidemiologic, case-mix, and utilization 
analyses are illustrated for diabetes mellitus. (JAMA 1984;251:637-644) 

DEVELOPMENT of scales measuring severity of diseases is a major challenge 
for the medical profession. Severity is defined here as the likelihood of 
death or residual impairment as the result of a disease, without 
consideration of treatment. Severity scales are useful in evaluating 
diagnostic efficiency of physicians, refining measures of prognosis, 
therapeutic effectiveness, and utilization of health care services, 
designing clinical trials, and developing systems for reimbursing hospitals 
based on the diagnoses of their patients 1 conditions. In the area of health 
care evaluation, severity scales enable more accurate measurement of health 
status before, during, and after medical intervention and comparisons among 
physicians, hospitals, and health insurance programs as to the types of 
patients being treated. (Ref. 1) This latter point has particular relevance 
in light of recent changes in the Medicare program. 

With the enactment of the Social Security Amendments of 1983, the 

Medicare program shortly will begin to reimburse hospitals on the basis of 
a fixed fee per case type. (Ref. 2) Regardless of the actual costs incurred 
in treating a given patient, within predetermined limits, the hospital will 
be paid a fixed amount. The system for classifying patients into payment 
categories is called Diagnosis Related Groups (DRGs) . The DRGs were 
developed by researchers at Yale University under support from the Health 
Care Financing Administration. (Ref. 3-4) The DRGs classify patients into 
470 categories based on principal diagnosis, principal surgical procedure, 
age, comorbidities, complications, and a few other factors. These 
categories were defined so as to group patients whose lengths of stay and 
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total charges are expected to be similar. However, there is no 
explicitmechanism in the DRGs to account for severity of illness within a 
particular diagnosis. 

Many physicians and hospital administrators fear that their 
institutions will be unfairly penalized under the new payment system 
because they believe that their patients are sicker than average and that 
these differences are not captured by the DRGs. This is especially the case 
for municipal, teaching, and large referral hospitals. Indeed, to the 
extent that substantial variation in severity is observed within any DRG 
category and to the extent that severity is correlated with resource use, 
hospitals face an incentive to admit only the least severe cases within 
that category. Should this occur, it will create widespread problems of 
access to inpatient care for many patients who need it most. Research is 
urgently needed to document the extent and nature of this potential problem 
by examining technically appropriate patterns of inpatient resource use by 
level of severity within each DRG category. 

This article describes an approach to measuring disease severity, 
called "staging," that can be used to address this problem and many others. 
Disease staging criteria have been developed for manual application to 
medical records and for automated application to computerized hospital 
discharge abstract data. These severity criteria cover practically all 
admissions to the typical hospital so that staging represents a 
comprehensive case-mix classification sytem. It also provides a means of 
assessing appropriateness of admissions and discharges, of examining 
associations between medical resource use and severity of specific 
diseases, and of comparing hospital outputs on both a large-scale basis and 
a case-by-case basis. The conceptual foundation of disease staging is 
described in this article, as well as the methods used to develop the 
severity criteria. Examples of potential applications of staging are 
presented, focusing on one disease, diabetes mellitus, to illustrate the 
usefulness of this measurement tool. 
ANALYTIC APPROACH%Definition of Disease 

To set the context for the discussion of disease staging, it is 
necessary to define the term disease. A disease is a well-defined model of 
a process of disruption in the normal homeostasis of 
psychological-physiological systems. Diseases should be distinguished from 
health problems and conditions, which are general descriptive attributes of 
psychological-physiological structure and/or functioning . Unfortunately, 
this is not always done by physicians, hospitals, health agencies, or 
insurance companies . The problem is compounded by the classification systems 
used to store, retrieve, and analyze health data. Health problems 
(symptoms, physical abnormalities, and pathological manifestations) are 
being classified, coded, and analyzed as if they were equivalent to 
disease. (Ref. 1-5) As a result, analyses of hospital output may be 
confounded and misspecified and have erroneous conclusions. 

A diagnosis should have information documenting the four elements 
required to define a disease: location of the problem, manifestations of 
the medical problem, cause of the problem, and severity of the problem. 
Each disease should be defined in terms of some specific organ or organ 
system involved. Thus, an infection is not a disease but a set of health 
problems. Note that the concepts of "health problem" and "disease" should 
be distinguished from "illness," which is defined as the actual 
manifestation of a disease in an individual patient. However, the focus of 
this article is on the concept of disease. 

Every disease must specify some characteristic pathophysiological 
change in the organ or organ system involved. Peptic ulcer would thus be an 
inadequate definition. Upper gastrointestinal tract bleeding secondary to 
peptic ulcer provides more specific information. 

An etiologic factor or set of factors causing the pathophysiological 
changes must be given. Neither pneumonia nor bronchial pneumonia would meet 
this criterion. Bacterial pneumonia would be an improvement, but optimally, 
the specific bacteria causing the pneumonia should be documented . Health 
problems such as congestive heart failure (CHF) that may result from a 
variety of causes would not be considered to be a disease even though CHF 
may be coded and may be the principal diagnosis appearing on many medical 
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records and discharge abstracts. The implication is that the physician did 
not know, even after study, or did not disclose what disease produced the 
CHF. 

Severity of the pathophysiological changes must be given, when 
severity is defined as risk of death or temporary or permanent impairment. 
This can be done simply through specification of the stage of the disease. 
According to this criterion, diabetes mellitus would not be a single 
disease but a family of diseases that includes hyperglycemia, diabetic 
acidosis, diabetic retinopathy, diabetic coma, peripheral neuropathy, and 
azotemia secondary to renal damage. Pneumococcal pneumonia with septicemia 
would be considered to be a different disease (severity) than pneumococcal 
pneumonia complicated by either empyema or meningitis. 

May diagnostic labels traditionally used by medical professionals and 
most diagnostic rubrics in the various coding systems (eg, International 
Classification of Disease: Clinical Modification, ninth revision (Ref. 6)) 
do not give these types of data. This may cause inaccurate evaluations of 
the effectiveness of medical treatment and biased analyses of hospital case 
mix. Patients are classified into inappropriate groups for comparison of 
utilization patterns, and overall severity of disease for any group of 
patients cannot be assessed. 

Disease Staging 

A method called "staging" has been developed for measuring disease 
severity. (Ref. 7.) The concept behind staging is borrowed from clinical 
medicine, specifically oncology. During the course of neoplastic diseases, 
there are discrete "stages" that can be defined and detected clinically, 
reflect the severity of the disease, and have clinical significance for 
prognosis and choice of therapeutic modality. This concept has now been 
applied to other medical and surgical problems to classify essentially all 
hospitalized patients. 

In staging, diseases are generically divided into categories of 
increasing levels of severity: stage 1, conditions with no complications or 
problems of minimal severity; stage 2, problems limited to an organ or 
system, significantly increased risk of complications over stage 1; stage 
3, multiple site involvement, generalized systemic involvement, poor 
prognosis; stage 4, death. Staging does not depend on actual utilization 
patterns or on expected response to therapy to assess severity. It is based 
on a conceptual model of the disease process itself rather than on the 
relative efficacy of medical technology. Note that even though disease 
technically ceases to exist on death of the patient, it is useful, from 
both heuristic and practical viewpoints, to include death as an end point 
of the severity scale. This provides explicit consideration of patients who 
die during the course of their treatments. 

METHODS 

Development of staging criteria for all known diseases is an 
exceedingly costly and probably unnecessary task. Therefore, a subset of 
conditions had to be identified. Two objectives governed this task: one was 
to include the major diseases in each etiology-body system class, the other 
was to cover the majority of admissions to a typical short-term general 
hospital. The first objective relates to the medical usefulness of the 
staging criteria, and the second relates to their usefulness for 
administrative, policy, and research purposes in hospitals. The 
etiology-body system matrix, as shown in the Figure, was used to classify 
the conditions that were important from either a medical or utilization 
perspective. The former was operationalized by one of the authors in 
consultation with medical colleagues. The latter was operationalized by 
analyzing diagnosis frequencies from a set of 387,000 hospital discharge 
abstracts. Finally, diagnostic coding manuals were inspected to ensure that 
omitted conditions were each potentially considered for inclusion in the 
list. Approximately 400 conditions were identified in this process. During 
the course of development of the staging criteria, other conditions were 
added by the physician consultants, some were redefined, and still others 
deleted. Finally, some "catchall" criteria sets were developed to handle 
residual cases when processing discharge abstracts. These steps ensured 
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that staging would be useful as a comprehensive case-mix classification 
system that could be applied to large groups of patients. 

Development of Medical Staging Criteria 

A panel of 23 medical consultants was formed to assist in 
specification of the medical staging criteria. (The panel members are 
listed at the end of the article.) Each disease was assigned to two members 
of the panel to be staged independently. The specific instructions to the 
panel called for every condition to be divided into at least four primary 
stage categories, according to the generic definitions specified 
previously. The panelists were encouraged to develop as many substages 
within each primary category as deemed appropriate. Each substage should 
place the patient at a significantly higher risk of morbidity or mortality 
or both than the previous substage and should be clinically dif f erentiable 
from other substages. When complications with each primary stage or 
substage cannot be differentiated in their severity, they should be 
classified as equivalent. Only single disease entities, without concern for 
complications arising from a simultaneous, unrelated condition (eg, skin 
infection in a patient with hypertension, or age of the patient), should be 
considered. Finally, stages of disease should be defined primarily in terms 
of biologic complications. 

The panelists were instructed as to the importance of specifying 
synonyms, so that the criteria could be made as general as possible for use 
with coded diagnostic data to capture all patients who may have the 
condition regardless of variations in use of medical terminology by 
physicians, nurses, and medical records technicians. 

Finally, the panelists were asked to furnish criteria for objectively 
validating the presence of the stage in the patient, either quantitatively 
or qualitatively, and to provide a list of references to support the 
medical judgments implicit and explicit in the criteria. 

Development of Coded Staging Criteria 

The medical medical criteria can be applied on a manual basis to 
medical records to analyze patients within an institution or a selected 
disease category. While this requires only a few minutes per patient , ^ which 
may be acceptable for physicians in recording diagnoses on patients 1 
charts, it may be too time-consuming and costly for retrospective use in 
large-scale research projects and utilization reviews. A computerized 
version of staging is required to facilitate historical analyses of large 
numbers of hospitalized patients. 

To accomplish this goal, a team of medical records technicians was 
employed to translate each stage and substage definition into diagnostic 
codes, using the three primary disease classification systems: ICDA-8, 
(Ref. 8) H-ICDA-2, (Ref. 9) and ICD-9-CM. (Ref. 6) Operationally, a 
procedure similar to that used for medical criteria development was used 
for the coding process. Each medical staging criteria set was coded 
independently and then reviewed by a third medical records technician to 
resolve discrepancies. When necessary, physician panel members were 
consulted to assist in the final decision making. 

Two types of problems had to be addressed in translating the medical 
criteria into coded criteria: lack of specificity in the coding systems 
themselves and lack of availability of certain data items on a typical 
discharge abstract. In terms of lack of code specificity, the problem is 
that coding systems do not always allow for the precision specified by the 
medical staging criteria. For example, the medical criteria for 
appendicitis classify localized peritonitis as stage 2.1 and diffuse 
peritonitis as stage 2.2. However, it is not possible to differentiate 
between localized and diffuse peritonitis in the ICDA-8 or H-ICDA-2 coding 
systems. This problem was resolved via a conservative strategy to understate 
stage of disease; ie, a patient with only the diagnostic codes for 
appendicitis and peritonitis would be classified as stage 2, since there is 
no way of knowing whether the peritonitis is localized or diffuse. Of 
course, if this patient had other complications of appendicitis, such as 
septicemia, then he or she is classified as stage 3. 

The other area in which changes to the medical criteria were necessary 
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relates to the lack of data (primarily physical findings and laboratory and 
roentgenographic study results) in most discharge abstract data systems. It 
is not possible to specify a stage (or a substage) that is defined solely 
on laboratory results by use of discharge abstract data. For example, the 
stages of aplastic anemia are defined in terms of hemoglobin levels and 
white blood cell and platelet counts. Again, the coded criteria tend to 
understate the severity of the disease in this situation. 

An example set of medical and coded staging criteria for diabetes 
mellitus is presented in Table 1. Diabetes was one of the more difficult 
diseases to stage because its manifestations range from mild to severe 
metabolic derangement, which are potentially reversible, and these may be 
superimposed on permanent dysfunction of other organs. 

Note: This table may be divided, and additional information on a 
particular entry may appear on more than one screen 
Table 1. — Medical and Coded Criteria for Disease Staging 
Diagnosis: Diabetes Mellitus; Etiology: Metabolic 



Stage 
1.0 

2.1 



2.2 



2.3 



Alternate Description 

or 

Synonym 

Hyperglycemia, "sugar dia- 
betes" 

Diabetes mellitus with com- 
plications of infectious 
nature: pyoderma, impe- 
tigo, f urunculosis , monil- 
ial vulvitis, monilial skin 
infection, cystitis, urethri- 
tis, epididymitis, prostati- 
tis, pyelonephritis 
Diabetes mellitus withDiabetes mellitus, infection, 
Septicemia and associated toxins of 

bacteria in bloodstream 
Diabetes mellitus with Diabetic acidosis or ketosis 



Common Description 
or 

Name of Condition 
Diabetes mellitus 

Diabetes mellitus with 
an infection in one or 
more systems (eg, 
skin, genital tract, 
urinary tract) 



acidosis 



2.4 Diabetes mellitus with 
retinopathy but with- 
out loss of vision, or 
glomerulosclerosis 
(without azotemia), 
or neuropathy (pe- 
ripheral or autonom- 
ic) , or gangrene (tis- 
sue breakdown) 
3.1 Diabetes mellitus with 
acidosis and coma, 
or retinopathy and 
loss of vision, or ne- 
crotizing papillitis, or 
azotemia 



3.2 



Diabetes mellitus with 
hyperosmolar coma 
3.3 Shock 
4.0 Death 
Diagnosis: Diabetes Mellitus; Etiology: 
IDC-9-CM Codes That Define 
Each Stage and Substage 
,10, 790.20, 250.00-250.01, 
250.80-250.91 
Stage 1.0+320.00-342.90, 
.00-245.10, 254.10, 289.20- 
.30, 420.00-422.99, 424.91, 
.89, 447.60, 480.00-486.00, 



Diabetes with ketonemia or 
ketonuria 
Microangiopathy 



Kimmelsteil-Wilson disease 

Arterial insufficiency with 
associated tissue break- 
down 

Diabetic coma with ketosis 
or acidosis 

Proliferative retinopathy 

Papillary necrosis, medul- 
lary necrosis 
Nonketotic, hyperosmolar 
hyperglycemic coma 



Metabolic 



Stages 
1.0 

2.1 



775. 



245. 
289, 
429. 
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510.00-510.90, 511.10, 513.00- 
513.10, 526.40, 566.00-567.90, 
569.50, 572.00, 577.00, 
580.81, 590.00-590.30, 595.00- 
595.40, 595.89-595.90, 597.00- 
597.80, 598.00-598.01, 599.00, 
601.00-601.90, 603.10, 604.00- 
604.99, 607.10-607.20, 590.90, 
608. 00, 608.40-608.81, 611.00, 
614.00-616.11, 616.30-617.90, 
680.00-68 6.90,711.00-711.99, 
728. 00,730.00-730.39, 730.80- 

730.99 

2.2 Stage 1.0+038.00-038.90 

2.3 Stage 1. 0+588.80, 791.60, 

276.20-276.40; 
250.10-250.11 

2.4 Stages 1.0-2.3+337.10, 362.18, 

443.81, 443.90, 446.60, 
447.10, 581.81, 785.40, 
354.00-356.90; 
357. 20, 362.01, 
250.40-250.74 

3.1 Stage 2.4+276.20, 369.00- 
369.90; Stages 1. 0-2.4+583.70, 

780.00, 790.60, 584.50-586.00, 
590.80-590.81; 362.02, 250.30- 

250.31 

3.2 250.20-250.21 

3.3 Stages 1.0-3.2+458.00, 458.90, 
%?785. 50-785. 59, 788.50, 994.00, 

994.80, 995.00, 995.40, 
958.40, 998.00, 999.40, 
669.10-669.14, 639.50 
4.0 Stages 2.2-3.3 + Death 



Computer Software System 

A computer software system has been developed to employ staging on 
large-scale data bases. An individual patient record from a computerized 
discharge abstract file is read by the program and systematically searched 
for the principal and associated diagnoses. If one of the conditions for 
which staging criteria exist appears among this vector of diagnostic codes, 
the patient is "staged" for that condition. For each such medical (or 
surgical) problem, patients are assigned the highest stage specifically 
justified when all of the diagnostic vector is taken into account; ie, the 
patient is staged for each diagnosis listed, given the other diagnoses 
listed. 

This software will classify patients into any of 420 diagnostic 
categories and assign an ordinal stage within each category. In addition to 
diagnostic data, the software also employs data on selected procedures, 
sex, and whether or not the patient was alive at time of discharge. 

Other than principal diagnosis, the staging software does not require 
any ordering of the diagnosis vector. Moreover, the software will overrule 
the principal diagnosis listed if it is a complication or manifestation of 
a secondary diagnosis, according to the medical criteria. For example, 
neuropathy is considered a manifestation of diabetes mellitus by the 
software when both conditions are listed on a discharge abstract, 
regardless of the order in which they appear. 

APPLICATIONS 

Staging can be done manually from medical records. Experienced 
medical records technicians can easily and reliably assign stage of disease 
from a full medical chart. Staging from the record gives more valid results 
than from the abstract or face sheet because of the greater amounts of data 
available . 
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Staging has been used in a number of studies to evaluate helath care 
quality and costs. (Ref. 10-11) It provides a useful framework for 
organizing clinical data on the patient for purposes of analysis of 
utilization patterns. A complete report on the development of the medical 
and coded staging criteria, together with the staging software, is now 
available to assist in development of new applications of staging. (Ref. 
12) 

One purpose of this article is to illustrate the uses of staging by 
means of three examples: epidemiologic, case-mix, and utilization analyses. 
Staging is a tool that can be used to address such questions as: Who tends 
to have more severe cases of a given disease? Which providers tend to treat 
more severe cases of a disease on average? How does utilization of health 
services vary with disease severity? 
Data 

The data for the empirical analysis presented herein were derived from 
discharge abstracts of 373 short-term general non-federal hospitals that 
are participating in the Hospital Cost and Utilization Project (HCUP) being 
conducted by the National Center for Health Services Research. (Ref. 13) 

Hospital Sample. — Hospitals were chosen with controlled selection 
sampling from a unverse that met minimum requirements related to data 
availability, quality, and continuity. An initial sample of 764 hospitals 
was drawn, of which 384 agreed to participate in the study and 373 provided 
discharge data for 1977. Comparison of the characteristics of the original 
sample with the final sample reveals a high degree of similarity. The final 
sample includes 65 proprietary hospitals and 71 public general hospitals; 
138 hospitals are located in rural areas, and 130 have less than 100 beds, 
while 103 have 400 beds or more. Hospitals with less than 25 beds were not 
included in the original sample. 

Patient Sample. — The volume of patient records contained in the 
discharge abstract files of 373 hospitals necessitated development of a 
sampling strategy for measurement of hospital use and case mix. A 
stratified random sample of abstracts was drawn from each of the 
participating hospitals for the calendar year 1977. The stratification 
variables were race (white or nonwhite), expected source of payment (Blue 
Cross, Medicare, Medicaid, commercial, self-pay, or other), single-multiple 
diagnosis, and time of year of discharge (first or second six months) . 

The total number of cases in patient sample is 392,456, with a range 
of 80 to 3,638 cases per hospital and a mean of 1,052 cases per hospital. 
Of these, only 275 cases did not contain sufficient data to be "staged" by 
the software, providing a total sample of 392,181 "staged" records. 

Other Data Sources. — Data on hospital characteristics, eg, size, 
teaching status, and location, were obtained from the 1977 Annual Survey of 
Hospitals conducted by the American Hospital Association. 

Results 

Diabetes mellitus was one of the ten highest-frequency conditions in 
the patient sample, with 1.5% of discharges, or 5,842 patients having this 
as their primary staged diagnosis. Almost half of these patients did not 
have any further complications related to diabetes listed on the abstract, 
so that they were classified as stage 1. On the other hand, the inpatient 
case fatality rate for this disease was 202 deaths per 10,000 cases. Almost 
20% of diabetes cases were admitted with an infection in one or more of the 
body systems listed as a major complication of the disease, while almost 
10% were admitted with acidosis as the major complication. Combinations of 
retinopathy, glomerulosclerosis, neuropathy, gangrene, necrotizing 
papillitis, and azotemia (with or without acidosis and coma) accounted for 
almost 20% of the cases. 

In the tables, the substages under stages 2 and 3 are combined so as 
to simplify presentation and obtain higher frequencies in each severity 
level to facilitate further breakdowns by patient and hospital 
characteristics. Chi-square tests were performed to test for presence of 
systematic associations between stage and patient and hospital 
characteristics. All of the results of these tests proved to be significant 
at the P =.05 level and below. 

Epidemiologic Analysis. — One question of interest can be addressed 
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with the staging criteria and software: who are the patients who tend to 
have more severe cases of a particular disease? In Table 2, stage 
distributions for diabetes by selected patient attributes are presented. 
One immediately remarkable finding is a strong association between stage 
and age. Older persons tend to have relatively more several cases of this 
disease. 

Note: This table may be divided, and additional information on a 
particular entry may appear on more than one screen 

Table 2. -- Distribution and Length of Stay (LOS) of Diabetes Mellitus 
Cases, by Stage of Disease and Selected Patient Characteristics, 1977 Rl 
Stage IStage 2 

Patient 

Characteristics No. %LOS, Days No. %LOS, Days 
All patients2, 91449. 9 R2 9.172.33039.9 R3 11.67 
Age, yr 

0-19 18950.5 R2 6.15 17647.1 R3 5.70 
20-44 R4 51049.3 7.19 44242.7 8.65 

45-641,10951.8 R3 8.88 82638.6 R3 11.79 
65-79 87048.8 R3 10.54 68738.5 R3 13.85 
>=80 23646.4 R3 12.21 19939.1 R3 15.89 
Type of admission 

Emergency R4 41742.5 10.39 42343.1 12. 97 
Regular2, 49751. 4 R3 8.971,90739.2 R3 11.36 
Surgical status 

Operated on R4 28325.8 8.38 67161.1 9.79 

Not operated on2. 63155.5 R3 16.641,65935.0 R3 16.27 
Source of payment 

Blue Cross R4 59852.0 8.47 46241.1 10.51 

Medicarel, 21247. 9 R3 10.75 97538.5 R3 14.02 
Medicaid 29951.1 8.28 23239.7 10.25 

Commercial 54955.0 R3 7.50 39439.5 R3 8.95 
Self-pay 10838.9 R3 6.80 14150.7 10.90 
Other 14849.5 8.91 12642.1 9.53 

Stage 3 Stage 4 

Patient 
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Patient 

Characteristics No. %L0S, Days 

All patients 5,842 100 10.47 
Age, yr 

0-19 379 100 R3 5.97 

20-44 R4 1,035 100 7.96 

45-642, 140 100 R3 10.24 

65-79 1,784 100 12.26 
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>=80 509 100 13.63 
Type of admission 

Emergency R4 981 100 11.71 
Regular 4,861 100 R3 10.22 
Surgical status 

Operated on R4 1,099 100 9.12 
Not operated on 4,743 100R3 16. 29 
Source of Payment 

Blue Cross R4 1,151 100 9.48 
Medicare 2,531100R3 12.33 
Medicaid 585 100 9.42 
Commercial 998 100 R3 8.19 
Self-pay 278 100 9.43 
Other 299 100 9.27 
(Ref. 1) Source: National Center for Health Serivces Research, 
Hospital Cost and Utilization Project. 

(Ref. 2) LOS is significantly different from LOS FOR REFERENCE GROUP 
AT P<.05 level. 

(Ref. 3) LOS is significantly different from LOS for reference group 
at P<.01 level. 

(Ref. 4) Reference group for significance tests of differences between 
means . 

Patients listed as emergency admissions on the discharge abstract are 
more likely to have higher severity and die than regular admissions, thus 
providing a validity check on the staging criteria. 

Fewer than one fifth of diabetes cases are operated on, but they have 
substantially higher severity levels, indicating an association between 
severity and resource use. This is also supported by a strong association 
between stage and number of procedures listed on the abstract. 

Finally, Medicare and self-paying patients are revealed to have 
significantly higher severity levels than other groups of patients. In the 
case of Medicare, this is likely to be age related. However, the higher 
severity of self-pay patients may be related to delay in seeking 
appropriate care, so that the disease progresses to a more advanced stage 
before care is initiated. This would be an example of where stage profiles 
can highlight potential problems in the provision of ambulatory care. 

Case-Mix Analysis. — A second application of staging is measurement 
of severity of hospitals' case mixes. By case mix, we mean the variety of 
different diseases being treated by the hospital. Following the definition 
of disease specified earlier, each stage of a disease constitutes a 
different disease that should be reflected in the case-mix measure. In this 
empirical example, the mix of case types within diabetes mellitus are 
examined. Staging can also be applied as a comprehensive case-mix measure 
for the entire hospital because the criteria cover almost all patients 
admitted to a typical hospital. 

Staging profiles for diabetes mellitus by selected hospital 
characteristics are presented in Table 3. Again, some interesting 
associations with severity are found. Proprietary hospitals are shown to 
admit less severely ill diabetic patients as compared with voluntary and 
governmental hospitals. Strong associations between bed size and severity 
are observed. Medical school-affiliated hospitals tend to admit relatively 
more severe cases than nonaffiliated hospitals. 

Note: This table may be divided, and additional information on a 
particular entry may appear on more than one screen 

Table 3. -- Distribution and Length of Stay (LOS) of Diabetes Mellitus 
Cases, by Stage of Disease and Selected Hospital Characteristics, 1977 Rl 

Stage IStage 2 

Hospital 

Characteristics No. %LOS, Days No. %L0S, Days 
All patients2,87949.8 R2 9.172,30940.0 R3 11.67 
Ownership 

Proprietary R2 30754.2 8.32 20135.5 10.72 

Voluntary2,15350.2 R3 9.521,70939.9 11.89 
State-local 40345.5 7.96 38143.1 11.22 

government 
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All Patients 

Hospital 

Characteristics No. %LOS, Days 
All patientsS, 777100 10.47 
Ownership 
Proprietary R2 566100 9.45 
Voluntary4, 289100 R3 10.73 



885100 



9.89 



02 
41 



State-local 
government 
No. of Beds 

25-99 R21, 113100 
100-2991,253100 
300-4991,922100 R3 11.16 
>=5001, 489100 R3 11.53 
Teaching status 
No affiliated with 
medical school R22, 884100 10.01 
Affiliated with 
medical school2, 856100 R3 10.95 
(Ref. 1) Totals within this Table may vary because of missing data. 
Total number of cases in Table 3 is less than Table 2 because 1% of the 
hospitals were randomly dropped when merging hospital characteristics to 
provide additional protection for institutional identities. Source: 
National Center for Health Services Research, Hospital Cost and Utilization 
Project . 

(Ref. 2) Reference group for significance tests of differences between 
means . 

(Ref. 3) LOS is significantly different from LOS for reference group 
at P<.01 level. 

(Ref. 4) LOS is significantly different from LOS for Reference group 
at P< .05 level. 

It is not appropriate to interpret these simple associations as 
causation. There are any number of potentially confounding factors. For 
example, proprietary hospitals in this sample tend to have fewer beds and 
to be located in the South and West, so that size and region are associated 
with ownership. Also, larger hospitals tend to have a higher likelihood of 



44 of 48 



4/22/01 10:29 AM 



DialogClassic Web(tm) 



http://www.dialogclassic.com/264656RB.HTML? 



medical school affiliation. These simple tabulations do serve, however, to 
illustrate that there are significant case-mix differences among hospitals 
with respect to severity for this diagnosis. 

Utilization Analysis. — An important hypothesis regarding hospital 
utilization is that sicker patients require more care. Staging enables a 
direct test of this hypothesis. Tables 2 and 3 also show how length of 
hospital stay varies with selected patient and hospital characteristics 
within each stage. This enables the dimension of severity to be held 
constant while the impact of other factors on length of stay is examined. 
Within each stage, t tests were performed to test for significant 
differences in lengths of stay by patient and hospital characteristics. A 
single group was selected for comparison purposes in the case of variables 
with multiple classifications, such as age and source of payment. 

Table 2 presents average length of stay breakdowns by stage and 
patient characteristics for diabetes mellitus.The first finding of interest 
is that length of stay increases with stage for this disease, ranging from 
9.17 days for stage 1 to 12.84 days for stage 4. The second finding of 
critical interest is that length of stay increases with age within stage, 
except for patients who die, where no statistically significant differences 
are found (in comparison with the 20- to 44-year-old age group) . 

Emergency admissions have longer stays for stages 1 and 2 than regular 
admissions. Surgical cases have considerably longer stays than nonsurgical 
cases of this disease for stages 1, 2, and 4. In the case of patients who 
die in the hospital, those who are operated on have almost three times the 
length of stay of those who are not. 

Length of stay breakdowns by source of payment for each stage of 
diabetes reveal associations with age, with Medicare patients having the 
longest stays and type of coverage, with commercial and self-pay patients 
having significantly shorter stays than Blue Cross patients for less severe 
stages of the disease. No significant differences in stays across pay 
sources are observed for stages 3 and 4, which suggests that length of stay 
becomes more variable with increased severity and also that economic 
factors may not be relevant at higher severity levels. 

Length of stay breakdowns for diabetes by selected hospital 
characteristics are presented in Table 3. Patients in voluntary hospitals 
are shown to have slightly longer stays for stage 1 than those in 
proprietary and state or local government hospitals. No differences by 
ownership are observed for stages 2, 3, or 4 . Patients in larger hospitals 
have longer stays than those in smaller hospitals for stages 2 and 3. The 
variability in length of stay for stages 3 and 4 precludes detection of any 
significant stay differences by ownership. Length of stay is associated 
with teaching activities for stages 1, 3, and 4. Patients in hospitals 
affiliated with a medical school tend to stay from three fourths to more 
than 4 1/2 days longer, on average, with the differential increasing with 
stage. Not shown here is that regional differences in length of stay are 
found for all stages except death. Patients in Northeastern hospitals tend 
to stay longer than patients in all other regions of the country, and those 
in Western hospitals tend to stay less time than all others. These findings 
are consistent with results for other samples. (Ref. 14-15) 

COMMENT 

A critical issue facing health care professionals is the development 
of techniques for measuring hospital output. Different diseases require 
different treatment modalities, including different types and magnitudes of 
professional and institutional resources to achieve desired outcomes. 
Methods must be developed that can be used to categorize patients in such a 
way that homogeneity within groups and heterogeneity between patient 
groups, with respect to the nature of their illnesses, are maximized. Such 
a classification scheme, which must allow for the assessment of severity of 
disease, can then be used for studies of hospital utilization and costs, 
development of equitable reimbursement mechanisms for hospitals, 
certification of institutions and providers for the treatment of different 
patient groups, and design of clinical trials to test the efficacy of 
different treatment modalities . The analysis presented herein reveals that 
diabetes is actually several diseases with differing patterns of care with 
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respect to length of stay, surgery, and overall service intensity . The 
association between severity and resource use provides predictive validity 
of the staging criteria. 

Considerable within-stage variation in resource use remains, much of 
which may be cause by factors other than biologic severity of illness. 
Patient-related factors, such as age, income level, expectations, health 
insurance coverage, family support, and presence of unrelated 
comorbidities; provider-related factors, such as specialty, degree of 
specialization, and choice of treatment modality; institutional factors, 
such as availability of special care units and extent of teaching 
activities; and community factors, such as supply of hospital beds, all can 
have significant impacts on cost and quality of care that should be 
considered along with stage of disease when analyzing these issues. (Ref. 
16-17) . 

Inaccuracies in recording and coding of diagnostic data on hospital 
discharge abstracts also may contribute to observed variation in resource 
use within stage of disease. Results of analyses of historical data files 
must be interpreted with caution because high within-stage variation in 
actual severity levels may be present. 

staging is relevant to the individual physician because it provides a 
tool for sharpening diagnostic classification. This will become 
particularly relevant in dealing with the impending regulations governing 
hospital admissions and discharges under the new DRG program for Medicare. 
As a consequence of the incentives in the DRG reimbursement system, new 
regulations governing use of the hospital are being developed. One outcome 
of this process will be greater requirements for documenting and justifying 
hospital admissions and discharges. Severity will play a crucial role here 
and staging can be used as an operational measure. 

In ' contrast to other existing casetype classification system such as 
the DRGs, an important characteristic of the staging criteria is their 
emphasis on medical meaningf ulness . (Ref. 18) Moreover, no utilization data 
are used to develop or operationalize the staging criteria. The DRGs use 
principal operating room procedures to classify patients in different 
groups and ignore etiology and severity of the problem. For example, 
patients receiving a coronary bypass procedure are classified differently 
than those with the same principal diagnosis but who received a nonsurgical 
therapeutic regimen. Diagnosis Related Groups are focused on what was 
actually done for the patient rather than on the nature of his disease. As 
a result, DRG No. 1 classifies patients on the bais of performance of a 
craniotomy and ignores whether patients have abscesses, aneurysms, or 
tumors. Comparisons between physicians and hospitals on patters of resource 
use in this DRG are likely to be misleading. Staging has an underlying a 
priori theoretical structure in which only clinically pertinent attributes 
of the patient are employed. This enables comparison and evaluation of 
surgery rates for a given stage of a disease across institutions and 
physicians. Perhaps more important is that staging enables incorporation of 
disease severity into the prospective payment system. 

CONCLUSIONS 

The utility of the coded staging criteria and the staging software has 
been demonstrated by the analyses of diabetes mellitus presented 
previously. Significant differences in stage profiles across selected 
patient and hospital characteristics were high-lighted. These can be used 
for purposes of conducting evaluations of quality of care in ambulatory 
care sytems and of appropriateness of admissions and discharges in 
hospitals . The latter point is particularly relevant under the new DRG 
payment system in which hospitals face incentives to maximize the number of 
"profitable" patients. Staging can be used to conduct assessments of the 
social epidemiology of disease and is also useful for analysis of inpatient 
resource intensity. Patient and hospital characteristics were shown to be 
significantly associated with length of stay within each stage of diabetes. 
While the purpose of this article is not to study diabetes per se, the 
analysis demonstrates the appropriateness of a disease-specific approach to 
utilization research. 

Disease stabing does not attempt to capture all of the variables that 
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may contribute to hospital costs or resource requirements. Indeed, it is 
specifically focused only on one — severity of disease. Thus, staging 
should not be asked to provide a complete explanation of variation in 
hospital resource intensity. Staging does provide the framework to ask what 
resources are necessary to respond to differences in the nature of the 
patient 1 s disease . 

Future work should focus on improving diagnostic coding systems and 
quality of discharge abstract data as well as on applying staging to 
medical records at different points during the hospitalization. Data should 
be analyzed at three crucial points: at admission, during the hospital stay 
(eg, the immediate postoperative period), and at time of discharge. This is 
required to document complications present when the patient was first seen, 
those developing during the hospitalization, and those successfully 
treated. Only by evaluating severity of patients' illnesses at various 
times can meaningful analyses of quality and cost-benefit tradeoffs be 
made . 
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